Method and system for optimized delta encoding

ABSTRACT

Methods and systems are provided for image processing. A plurality of correlation parameters representing degrees of correlation between two or more images of a plurality of images may be produced. An optimized correlation dependency graph may be produced according to the plurality of correlation parameters. The plurality of images may then be delta encoded according to the optimized correlation dependency graph. For example, the optimized correlation dependency graph may be used for performing a correlation encoding operation. The plurality of correlation parameters may be produced, for example, in accordance with one or more correlation metrics associated with the correlation encoding operation.

CROSS-REFERENCE TO RELATED APPLICATIONS/INCORPORATION BY REFERENCE

This application claims priority to U.S. application Ser. No.13/729,346, filed Dec. 18, 2012 and European patent application number11196029.0, filed Dec. 29, 2011, which is hereby incorporated herein byreference in its entirety.

FIELD

The invention relates to the field of data encoding, for example forcompressing data for transmission or storage. In particular, but notexclusively, it relates to delta or correlation encoding techniques.

BACKGROUND

Differential or delta encoding is commonly used in data processing, andis particularly useful for compressing strongly correlated, ordered datasets, such as sequences of video images. Since there is usually a strongcorrelation in content between the successive images in a videosequence, it is possible to achieve significant reduction in datatraffic by only transmitting the difference between one image and theprevious image, instead of transmitting each complete image.

Other applications for delta encoding include data backup systems, inwhich, instead of storing a complete new copy of the data to be backedup, an incremental backup can be created, in which only the differencesto the previous backup are stored for the new backup.

Delta encoding techniques have also been proposed as a substitute, orsupplement, for web page caching. In this case, a web-page may be cachedlocally by a browser, for example, and then, instead of refreshingentire web-pages when necessary, the browser and web server negotiatesubsequent transactions such that only incremental changes to web-pagesare communicated to the browser. Similarly, websites which are to bereplicated (mirrored) in order to improve their accessibility andreliability, can be kept synchronized with each other, by exchangingonly the (delta) content between the mirrored sites.

Correlation encoding may be lossless, in which case data is encoded insuch a way that it is subsequently completely reconstitutable to itsoriginal state, or lossy, in which case certain approximations are madeduring the encoding process, with the result that the encoded data nolonger contains all the information required to reconstitute the data inits original state. In general, lossy encoding offers a significantlygreater compression rate than lossless encoding.

Modern mobile phones are capable of taking many photographs anduploading the photos via wireless internet connections. One reason whymobile phones and smartphones have become a major image capture deviceis that the quality of the photos taken with these cameras isincreasing. There are many web-based services which allow users toarchive and share their private photos. However, the necessary accessbandwidth may be unavailable (or too expensive) to permit frequentuploading and storage of photos automatically in an internet-based photomanagement service, for example.

A problem for network based photo services (e.g. photo archives or photosharing services) is that the upload of images may take a long timebecause of the large size (resolution) of the images. For archivingservices the user usually wants to store the best quality available (rawimages with high resolutions in space and color domain). With thelimited upload bandwidths available using DSL or mobile networks, theprocess of image transfer can take a long time. In some cases theprocess may need to be scheduled in advance, and may take many hours.

Data sets, such as batches of images, are routinely compressed, forexample when the data sets are to be archived or transmitted, and whereit is important to reduce the amount of bandwidth or data storage spacerequired for the transmission or storage of the data sets. Generalpurpose data-compression algorithms, such as the well-known Lempel-Zivalgorithm and its successors, involve identifying recurring patterns ofdata in a batch of data, and building a dictionary of such patterns,such that each pattern can be referred to by its dictionary reference.

The term “data set” is used in this application to refer to any item ofdata which may be the subject of correlation processing with other itemsof data—for example processing for compression, analysis or other typesof data manipulation. Several examples of such data sets are given inthis application, such as a collection of photographs in a digitalcamera, which are to be uploaded to a web-server. Another example wouldbe batch-processing of the content of a batch of digital images—if forexample a user wishes to carry out an image enhancement operation suchas sharpening or contrast-enhancement on all the images in the batch, orconvert a batch of images from one color space to another. Batchprocessing in this context means performing the same operation, or thesame type of operation, on a batch (plurality) of data-sets.

Another example of data sets is in the batch-processing of OCR (opticalcharacter recognition) documents. Text documents can be subjected to acorrelation encoding process, for example, in which similar pieces ofscanned text (phrases, words or word segments, for example), which occurmultiple times in the scanned images, are encoded as references toentries in a dictionary. In this case, each data set may be a wholescanned image (e.g. a page), or it may be a section of an image, such asa part of a text document which has been identified as a word or othergroup of characters or symbols.

Another example of data sets might be a collection of diverse computerfiles in a directory. Groups of such files may be collectively subjectto various forms of processing such as data compression, data backup,virus-checking, file-system defragmentation, synchronizing etc.

It is known to use general-purpose data-compression algorithms tocompress such unordered data sets. Files, or batches of files, can becompressed using the ubiquitous Zip algorithm, for example, whichlosslessly generates a compressed file containing all the informationrequired to reconstitute the original file or files.

It is often necessary to process such data sets in batches. For example,a folder of files may need to be copied in one operation from acomputer's internal storage to an external storage device such as a USBstick. Or a batch of holiday photographs may need to be uploaded from amobile phone (for example via a wireless mobile network) to asocial-media website server. Or a collection of pages of text may needto be scanned and OCR'ed in one operation.

Each of these operations can be speeded up by reducing the amount ofinformation to be processed. In the case of compressing batches of datasets for transfer, conventional methods either compress the data setsindividually, in which case the data sets can be transferredindividually but the compression is sub-optimal, or they can becompressed as one file, in which case the compression is improved butthe files cannot be transferred individually. Similarly, in the case ofthe batch OCR example, the amount of processing to be carried out can bereduced by encoding the whole batch of pages. The larger the data sampleto be encoded, the more efficient is the encoding, since the likelihoodof similar patterns (and therefore greater correlation) recurring isgreater.

Further limitations and disadvantages of conventional and traditionalapproaches will become apparent to one of skill in the art, throughcomparison of such systems with the present invention as set forth inthe remainder of the present application with reference to the drawings.

BRIEF SUMMARY

Aspects of the present invention are aimed at reducing or eliminatingthe above problems with the prior art batch processing techniques, forexample for unordered or arbitrary data sets. In particular, but notexclusively, aspects of the invention are directed to enabling areduction in the overall amount of data to be processed, while stillretaining the ability to process the data sets, or groups of data sets,individually.

To this end, aspects of the invention comprise a method of preparing abatch of data sets for a batch processing by a predetermined correlationencoding operation, the method including an analysis step of analyzingthe batch of data sets to determine a plurality of correlationparameters representing degrees of correlation between member data setsof said batch of data sets, and an optimization step of using theplurality of correlation parameters to generate an optimized correlationdependency graph of the member sets performing the predeterminedcorrelation encoding operation.

According to a variant of the method of the invention, the analysis stepdetermines the correlation parameters in accordance with one or morepredetermined correlation metrics for the predetermined correlationencoding operation.

According to a variant of the method of the invention, the methodcomprises a pre-analysis step of determining in the batch of data setsan optimized data subdivision and/or an optimized correlation metric forperforming the correlation analysis step.

According to a variant of the method of the invention, the correlationoptimization step generates an optimized correlation dependency graphwhich maximizes the overall correlation parameter values between thedata sets.

According to a variant of the method of the invention, the methodfurther comprises the predetermined correlation encoding operation, inwhich the batch of data sets is correlation-encoded in a sequencedetermined by the optimized correlation dependency graph.

According to a variant of the method of the invention, the correlationencoding operation generates a plurality of correlation-encoded deltasets, for performing a predetermined data set processing operation.

According to a variant of the method of the invention, the methodcomprises a delta encoding step, wherein the delta sets are compressedand/or encoded for the data set processing operation.

According to a variant of the method of the invention, the deltaencoding step performs a further correlation analysis and/or correlationoptimization step on the delta sets.

Aspects of the invention also envisage a system for preparing a batch ofdata sets for a batch processing using a predetermined correlationencoding operation, the system performing: 1) correlation analysis,wherein a processor is used for analyzing the batch of data sets todetermine a plurality of correlation parameters representing degrees ofcorrelation between member data sets of said batch of data sets; and 2)correlation optimization, wherein the processor uses the plurality ofcorrelation parameters to generate an optimized correlation dependencygraph of the member sets for performing the predetermined correlationencoding operation.

According to a variant of the system of the invention, the correlationanalysis is adapted to determine the correlation parameters inaccordance with one or more predetermined correlation metrics for thepredetermined correlation encoding operation.

According to a variant of the system of the invention, the processordetermines an optimized data subdivision and/or an optimized correlationmetric for performing the correlation analysis step.

According to a variant of the system of the invention, the correlationoptimization is adapted to generate an optimized correlation dependencygraph which maximizes the overall correlation parameter values betweenthe data sets.

According to a variant of the system of the invention, the processorperforms the predetermined correlation encoding operation on the batchof data sets in a sequence determined by the optimized correlationdependency graph.

According to a variant of the system of the invention, the correlationencoding operation is adapted to generate a plurality ofcorrelation-encoded delta sets, for performing a predetermined data setprocessing operation.

According to a variant of the system of the invention, the processorperforms a delta encoding for compressing and/or encoding the delta setsfor the predetermined data set processing operation.

Various advantages, aspects and novel features of the present invention,as well as details of an illustrated embodiment thereof, will be morefully understood from the following description and drawings.

BRIEF DESCRIPTION OF SEVERAL VIEWS OF THE DRAWINGS

The invention will now be described in greater detail with reference toaccompanying figures, in which:

FIG. 1 shows an example of data sets (pictures) to be processed.

FIG. 2 shows a schematic view of a system according to a firstembodiment of the invention.

FIG. 3 shows a schematic view of a system according to a secondembodiment of the invention.

FIG. 4 shows a schematic view of a system according to a thirdembodiment of the invention.

FIG. 5 shows a schematic view of a system according to a fourthembodiment of the invention.

The figures are given by way of example only, and are not intended tolimit the scope of the invention, which is set out in the appendedclaims. Where the same reference signs have been used in differentfigures, these reference signs are intended to refer to the same orequivalent features.

DETAILED DESCRIPTION

FIG. 1 shows an example of a batch of data sets—photographs 31 to 38,which may for example be stored in the memory of a camera. The sequenceof the data sets is arbitrary and has no significance for batch dataprocessing purposes. The content of each data set (image) in thisexample is not systematically related to the content of other data setsin the batch. If the camera user wishes to transfer the eight files viaBluetooth® to his or her laptop, or upload them to a web-server via amobile wireless connection, it is irrelevant to him or her in whichorder the images are transmitted.

In prior art systems, the photographs would either be transferred asindividual files (possibly having been individually compressed), or theymight be compressed as a batch and transferred as one compressed file.

By applying principles of the present invention, on the other hand, theimages of FIG. 1 could be correlation-encoded before transferring inorder to reduce the total amount of data transferred (and hence reducethe required bandwidth and/or transfer time for the transfer). However,the photographs are not strongly correlated, as would usually be thecase with a sequence of video images. Nevertheless, there arecorrelation relationships between the photographs, and these correlationrelationships can be exploited to achieve a significant reduction in theamount of data to be transferred. Photographs 31 and 36, for example,have similar backgrounds (hills) and contain similar image features(people and dog). Photographs 33 and 38 share the same road andbackground, while the beach scenes of photographs 35 and 32 are verysimilar except for the intensity levels of the foreground. Photos 37 and34 share little content, but the pixels and pixel patterns which make upthese images may share common characteristics which can be used in acorrelation encoding process to significantly reduce the amount of dataredundancy between the two photos.

The optimization process may comprise maximizing the degree ofcorrelation between images in the batch. The term “correlation” is usedhere in a broad sense. In a simple case, the term may be taken to referto a straightforward correlation between two variables, or two sets ofvariables, such as the pixel intensity patterns of two different images.Or it may be understood to mean a more complex metric such as amultidimensional correlation mapping in which the mapping vectorcomprises multiple partial correlations, each partial correlation beingbetween two variables. Alternatively, the term may be understood to meana multiple correlation between multiple variables, representable forexample as a linear regression among n variables in an n-dimensionalspace.

The terms “correlation analysis”, “correlation optimization” and“correlation processing” are used in this description to describedistinct process steps. Correlation analysis and optimization areinitial steps. The term “correlation processing” is used to refer to asubsequent step, such as delta encoding, which is subsequently performedon the batch of data sets which have been ordered as a result of thecorrelation analysis and optimization steps. Where the term “subsequentprocessing” is used, it refers to the correlation processing (e.g.,correlation or delta encoding) or to the combination of correlationprocessing and a further step such as transfer or storage or otherprocessing which may be performed on the batch of correlation-processed(correlation-encoded) data-sets (images).

In the simple case described below, the correlation analysis andoptimization may use an unvarying metric (all correlations between pairsof images, for example, are calculated using the same correlationmetric). However, it is also possible to vary the metric during theprocess (dynamically, for example, in response to analysis resultsdetermined during the process, or according to a predetermined variationscheme) in order to achieve an optimized correlation measure for thebatch of data-sets or for particular sub-sets of the batch of data-sets.

The correlation optimization process may also comprise maximizing anoverall (total) amount of inter-data-set correlation within the batch.This step is sometimes referred to as minimizing the entropy of a batchof data sets. The efficiency of the subsequent processing stronglydepends on the order (or dependency arrangement) in which the data-sets(images) are processed, as will be seen later. A predefined correlationmetric, selected as suitable for the subsequent correlation processingto be performed on the batch of data sets, can be used in thecorrelation analysis and optimization steps to determine the correlation“distances” or degree of correlation between data sets (e.g. images).Where the subsequent processing comprises batch image transfers, such acorrelation metric might for example indicate the degree of similaritybetween pairs of images. Once all correlation parameters between thepairs of images in the batch have been determined, an optimized imagesequence can be found by ordering or arranging the images in a way whichwill maximize the efficiency (minimize transfer time, for example) ofthe subsequent processing.

In a simple variant, the degree of correlation between two given imagesmay be treated as a constant for a given correlation metric, and thecorrelation parameters could in this case be represented as a matrix ofcorrelations between pairs of images. However, a further improvement tothe correlation optimization may in some cases be obtained by varyingthe correlation metric during the correlation analysis in order toachieve a yet greater optimization of the degree of correlations betweenthe data sets.

FIG. 2 shows in schematic view an example of a system according to anembodiment of the invention. In the example shown, data sets 1-11,collectively referred to as batch 15, represent an arbitrary collectionof data sets with unknown intercorrelation between the data sets. Theymay be photographs, as in FIG. 1, for example. The data sets areanalyzed for correlation by correlation relationship analysis 20, whichbuilds a set of correlation parameters 22, each correlation parameterrepresenting the degree of correlation between data sets. In a simpleimplementation of the system, the correlation parameters may representpairwise correlations between pairs of data sets, but in otherimplementations the correlation parameters may also be betweenpermutations of multiple data sets. In analyzing correlationrelationships between data sets, the correlation relationship analysis20 may refer to correlation metric information 21, which can be used todefine or control the correlation analysis process. The correlationmetric information 21 may impose restrictions or priorities, forexample, on the kinds of correlation which are to be detected ormeasured between data sets. Thus, for example, in a digital camera inwhich all images share the same size and color space; thesecharacteristics may be excluded from correlation analysis. For greyscaleimages, the correlation control information 21 may specify that colorparameters should be ignored, or (for example if the greyscale imagesare scanned text documents) that vector patterns are to be given ahigher weighting than, say, local raster pixel patterns. The nature ofthe correlation metric information 21, and therefore the nature of thecorrelation analysis performed by correlation relationship analysis 20,may depend largely on the type of encoding process which is to beperformed subsequently. The correlation metric information 21 may alsobe amended to take into account the results of correlation analysisand/or optimization operations which are carried out. In this way, thecorrelation metric information 21 can act as a knowledge base,accumulating rules, parameters and other information which can be usedfor defining or controlling future correlation analysis operations.

In the example shown, the correlation analysis 20 produces a set 22 ofcorrelation parameters which represent correlations, according to rulesspecified by correlation control information 21, between the variousdata sets 1 to 11. Each of these correlation parameters may specify, forexample, the type(s) of the correlation, as well as specifying whichdata sets the correlation was found amongst, and the degree of thecorrelation (which may be expressed for example as an objective or arelative value).

Once the correlation analysis has been performed by the correlationrelationship analysis 20, the resulting correlation parameters 22 arethen used by correlation optimizing 23 to generate an optimizeddependency graph 16, which defines an optimized pattern for performing asubsequent processing (not shown) of data sets 1-11. Correlationoptimizing 23 may call on additional correlation optimization parameters(rules, heuristics etc.), which may be predetermined for a particularapplication or a particular type of batch 15 of data sets 1-11.

In the example shown in FIG. 2, the optimized encoding graph 16 is alinear sequence which starts with data set 7 and proceeds through 6, 11,10, 9 etc. to data set 1. The subsequent correlation encoding of datasets 7 through 1 in the sequence 16 may comprise: take data set 7 as itis, then determine the set 6′ of delta values for data set 6 (thecorrelation parameters which relate data set 6 to data set 7), thendetermine the set 11′ of delta values for data set 11 (the correlationparameters which relate data set 11 to data set 6), and so on, until thelast set 1′ of delta values is determined for the sequence 16.Subsequent processing may then be carried out on data set 7 and deltasets 6′ to 1′, in place of the original data sets 7 to 1, in sequence16.

Note that the term “optimized” is used in this context to mean improved(i.e., changed towards an optimum state). The term “optimized” does notnecessarily mean that a particular optimum state is reached.

In the particular example of uploading a set of images to a web server,the first image 7 in the optimized sequence 16 could be uploaded as itis (or compressed using a known compression technique and thenuploaded), while the remaining images 6 to 1 need not be transferred.Instead, their corresponding sets of delta values 6′ to 1′ could besuccessively uploaded. Optionally, the delta sets 6′ to 1′ could also beindividually compressed, or they could be subject to a similarcorrelation optimization process as for the original data sets (givingrise to a multi-pass or recursive correlation-optimization process).

In such a multi-pass or recursive correlation analysis/optimizationprocess, the delta sets produced by one particular correlationanalysis/optimization process may themselves be subject to a furthercorrelation analysis/optimization process, thereby enabling a yetgreater degree of correlation encoding.

There are many types of correlation metric which may be taken intoaccount when optimizing the correlation between data sets. In the caseof images, for example (FIG. 1), correlation may be based on one or moremetrics—temporal, spatial, contour, vector, color, size, format, etc.Other parameters, such contextual parameters, may also be used. Examplesof contextual parameters are given in the “Parthenon” example later inthis description.

Furthermore, the correlation analysis need not simply be between twodata sets, but could include clustering or correlation across differentpermutations of multiple data sets.

Note that correlation relationship analysis 20 and correlationoptimizing 23 may be dedicated hardware units (e.g. electroniccircuits), or they may be implemented as software in a computing unit.One or more neural networks may be used, for example to implement aKohonen-type mapping from a set of predictor parameters (informationknown about the batch 15 and/or about individual data sets 1-11, forexample) to a correlation space, in which the correlations between datasets can be more quickly and/or more effectively determined.

Referring back to FIG. 1, it can be seen that a correlation optimizationprocess as described above might generate an optimized sequence, forexample, of 31, 36, 37, 34, 32, 35, 33, 38, merely taking into accountthe visible features of the photographs. In practice, the correlationanalysis may be performed in a much more detailed fashion, comparingpixel patterns, vector patterns, etc. using any of the many known imagecorrelation techniques.

A rather simple example has been used to illustrate one embodiment theinvention, but in fact the correlation optimization process may besignificantly more complex in other embodiments. A pre-analysis step maybe included, for example, to analyses the batch 15 of data sets 1-11 todetermine an optimum collection of data sets (which may be divideddifferently from the original data sets 1-11) and to identify optimizedsizes or formats of the data sets on which the actual correlationanalysis will be performed. In the case of the example of FIG. 1, forexample, a pre-analysis step might determine that the correlationanalysis step would be more efficient if the batch of images weredivided up into blocks of pixels of a particular size. Or thepre-analysis step might determine that the correlation analysis stepwould be more efficient if the batch of images were divided up byextracting certain macro-features (e.g. dog, beach, palm, hills,mountain etc.) and treating each of the extracted macro-features as adata set in its own right. Macro-features identified in the pre-analysisstep may be stored for later use in the subsequent processing.

Such pre-analysis, correlation analysis and optimization steps may becomputationally demanding, especially in small portable devices such asmobile phones or digital cameras. For this reason, one or more of thesesteps may be performed during a down time of the device. It is alsopossible to use heuristic methods for one or more of these steps. Ifiterative techniques are used for the pre-analysis, correlation analysisand/or optimization steps, then these steps can be performed in thebackground or during down time until a predetermined amount of analysisand/or optimization has been performed.

Once the correlation analysis and/or optimization steps have determinedwhich data set(s) should be processed first (i.e. processed withoutcorrelation encoding), the subsequent processing of this/these data setscan begin. In this case, the analysis and/or optimization of theremaining data sets can continue while the first data set(s) is/arebeing processed. In this manner, the method makes full use of the timeavailable for achieving an optimized correlation dependency graph. Theanalysis and optimization can continue until there are no more deltavalue sets to analysis/optimize (e.g. when the processing of thepenultimate delta set has begun, and only the last delta set remains tobe processed.

If the correlation processing of the analyzed/optimized batch isinterrupted, the processing need not start from the beginning again, butcan simply continue from the point it had reached when it wasinterrupted. The processing can either continue with the nextunprocessed delta set, or it can take the next unprocessed original dataset as a starting point. Furthermore, the data set and delta sets whichhad been transferred before the interruption can be treated asfully-processed (e.g. fully transferred) data sets. This is in markedcontrast to the “all or nothing” operation of some prior art methods ofprocessing compressed data sets (e.g. zipping of batches of imagefiles), in which the entire zipped file must be processed (e.g.transferred) before any of the individual data sets can be regarded asfully processed.

FIG. 3 shows an analysis/optimization procedure similar to that of FIG.2, except that the resulting optimized correlation dependency graph 16is not a linear sequence but a branching structure, branching at datasets 6 and 9. Such optimized dependency graphs may have virtually anydependency structure.

The correlation dependency graph 16 shown in FIG. 3 has only one first(or starting) data set, 7, as with the correlation dependency graphshown in FIG. 2. However, there may be instances where the correlationanalysis/optimization process determines that the subsequent processingwill be more efficient if the data graph comprises multiple first datasets. In this case, the dependency graph(s) generated by the correlationoptimization 23 will have more than one “starting” data set, each ofwhich will need to be processed as a complete data set, and each ofwhich may have other delta sets dependent on it.

In the above examples, the correlation optimization is described asbeing carried out on the basis of an analysis of (only) the data setswhich are to be batch processed. However, the correlation optimizationmay also take into account data sets which are not part of the batch tobe processed. This variant presents at least two possible advantagesover the simpler case: it can improve the correlation optimization, andit can in certain cases obviate the processing of “starting” data sets,with the result that all of the data sets in the batch of data sets canbe processed in their delta/correlation encoded form.

Generally, the more data sets are taken into account when calculatingthe correlation optimization, the greater will be the degree ofcorrelation optimization, and the more efficient will be the subsequentcorrelation encoding of the data sets for processing.

FIGS. 4 and 5 illustrate variants of this embodiment in schematic form.FIG. 4 shows how the embodiment illustrated in FIG. 2 can be adaptedsuch that correlation optimization is performed not merely on the batchof data-sets 15, but also on a plurality of reference data-sets 41 to56, which are collectively indicated as 64. The correlation optimizationcalculations may be carried out by correlation relationship analysis 20and/or by reference correlation relationship analysis 60, usingcorrelation metric information 21 which can be used to define or controlthe correlation analysis processing performed by correlationrelationship analysis 20 and/or by reference correlation relationshipanalysis 60.

The correlation analysis processing results in produces sets 22 and/or62 of correlation parameters which represent correlations, according torules specified by correlation control information 21, between thevarious data sets 1 to 11 and/or between data sets 41 to 56, in a mannersimilar to that described earlier in this description. Referencecorrelation parameters 62 may be pre-generated, based on the referencedata sets 64, or they may be generated on the fly, in dependence of thecorrelation analysis being carried out on the batch 15 of data sets 1-11to be batch processed.

The correlation parameters 22 and/or 62 may then be used by correlationoptimizing 23 and/or 63 to generate an optimized dependency graph 16,which defines an optimized pattern for performing a subsequentprocessing (not shown) of batch 15 of data sets 1-11 and reference datasets 64. In the example shown in FIG. 4, the optimized encoding graph 16is a linear sequence which starts with data set 48 and proceeds through6, 11, 10, 9 etc. to data set 1. Reference data set may for example be adata set which has already been subjected to correlation processing (forexample, it may be an image which has already been uploaded, or adata-set which has already been compressed). In the illustrated case,data sets 1 to 11 have been reordered in a sequence 6, 11, 10, 9, 8, 5,4, 3, 2, 1, 7 which is optimized for a subsequent processing operation.In this case, the correlation analysis processing determined thatreference data set 48 offered an optimized starting data set. Thesubsequent processing can then be performed on the sequence of data sets48, 6, 11, 10, 9, 8, 5, 4, 3, 2, 1, 7. The sequence ofcorrelation-processed data sets would then be 48, 6′, 11′, 10′, 9′, 8′,5′, 4′, 3′, 2′, 1′, 7′.

In some cases it may not be necessary to perform the correlationprocessing on the reference data set 48, in which case only the sequence6, 11, 10, 9, 8, 5, 4, 3, 2, 1, 7 need be subjected to correlationprocessing. The sequence of correlation-processed data sets would thenbe (as illustrated in FIG. 4 dashed lines) 6′, 11′, 10′, 9′, 8′, 5′, 4′,3′, 2′, 1′, 7′.

FIG. 5 shows how the embodiment illustrated in FIG. 3 can be adaptedsuch that correlation optimization is performed not merely on the batchof data-sets 15, but also on a plurality of reference data-sets 41 to56, which are collectively indicated as 64. As will be understood, thisvariant is analogous to the adaptation illustrated in FIG. 4, with theexception that a more complex graph structure 16 is generated,comprising multiple “starting” data sets.

At some stage in the procedure, the data sets 1-11 in the batch 15 maybe added to the reference data sets 64. In this way, the number ofreference data sets 64 can be increased to give improved reference datafor future correlation analysis processing. Correlation parameters 22,24, 62, 64 may also be accumulated in a knowledge base and madeavailable for future correlation analysis and/or correlationoptimization processing.

The above variants will now be illustrated using an example of how themethod of the invention can be practically implemented.

Taking the example of uploading a batch of photos to a server (forexample a webserver), it is possible to use characteristics of imageswhich are already on the server in order to improve the correlationoptimization process. It may for example be the case that a number ofimages are stored both in the local device (e.g. camera or mobile phone)as well as on the server (some of the images may previously have beenuploaded, for example). In such a case, the correlation optimization canbe performed locally—not only on the batch of images to be uploaded, butalso on other locally-stored images, which are known to be also storedon the server. This variant has two additional advantages: firstly thecorrelation optimization is performed across a larger number of imagesthan merely the batch of images to be processed, and therefore resultsin an improved set of correlation parameters for the subsequentprocessing and uploading. And secondly, the correlation optimization canbe performed such that the “starting” image(s) is/are selected to be oneof the images already on the server; in this case, it is not necessaryto transfer the starting image(s) to the server at all, and all of theimages in the batch to be uploaded can be uploaded as correlationencoded data.

When the encoded images are uploaded, they can be accompanied orpreceded by a parameter set which includes all the information requiredfor decoding the uploaded images once they arrive at the server. Thisparameter set may for example include an indication of which image(already on the server) should be taken as the starting image, anddetails of the encoding (e.g. correlation function) which was used toencode the images in the batch. The server can then use this informationto decode the images, whereupon the uploaded images can also be added tothe number of images which is available for future correlationoptimization processing of images.

It is thus possible to avoid processing any full data sets (uploadingany whole images), and this can significantly reduce the total amount ofdata to be processed (uploaded). This method can also be used for a“batch” which includes just a single data set (image)—in this case,correlation optimization can be performed on a plurality of images whichincludes the single image itself and at least one image which is knownto be already on the server, to determine an optimized selection ofcorrelation parameters to use in encoding the single image foruploading. The optimized correlation parameters can include theselection of a reference (“starting”) image and/or the selection of anoptimal correlation function, for example.

The above example describes a case in which correlation optimization isperformed using a) the batch of data sets (images) to be processed(uploaded from the local device to the server) and b) a separate groupof one or more other data sets (reference images) which are on both thelocal device and the server. However, it is also possible to adapt themethod for use in situations where these “reference” images are on theserver but are not necessarily available locally, in which case thecorrelation optimization can be performed on a) the batch of data sets(images) to be processed (uploaded from the local device to the server),as above, and b) a separate group of one or more other data sets(reference images) which are on the server. In this case, thecorrelation analysis processing can be partitioned into at least a firstcorrelation analysis processing, in which the reference images on theserver are correlation-analyzed on the server, and a second correlationanalysis processing, in which the batch of images to be uploaded arecorrelation-analyzed on the local device. The second correlationanalysis processing may use the results of the first correlationanalysis processing in order to achieve further optimized correlationparameters (e.g. starting image, correlation function) which can be usedfor the subsequent encoding and processing. Again in this way it ispossible to select a starting image which is already on the server, thusobviating the need for uploading any whole (unencoded) images.

Depending on the particular application of this method, some or all ofthe reference data sets 64, the reference correlation relationshipanalysis 60, the reference correlation parameters 62 and referencecorrelation optimizing 63 may be implemented temporarily and/orspatially remote from the data sets 15, the correlation relationshipanalysis 20, the correlation parameters 22 and/or correlation optimizing23. Thus, while the correlation relationship analysis and/or thecorrelation optimization may be computed in a local computing unit (notshown), the reference correlation parameters 22 may be pre-computedand/or computed on a remote server unit (not illustrated), accessed forexample via a wireless, mobile or wired network.

It is of course desirable to avoid the need for transferring largequantities of data between the local and remote computing units in sucha case. However, if the local correlation analysis 22 is to be able torefer to the reference data sets 64, it requires access to someinformation about the remote reference data sets 41 to 56. This problemcan be overcome, for example, by pre-computing a set of correlationanalysis information and making it available for access by the localcorrelation analysis 22. Alternatively, abstracted versions of thereference data sets may be made available. This abstraction can be asimple data-reduction (e.g. a reduction in resolution if the data setsare images), or it can be a more intelligent analysis of the data setsto extract representative patterns or elements from the data sets. Theabstraction techniques could also be combined.

In this way, correlation analysis 22 can have access to a large numberof reference data sets 64, and thereby greatly improve the correlationencoding of the batch 15 of data sets 1 to 11 which can be achieved.

A similar method can also be implemented in order to make informationabout the batch 15 of data sets 1-11 available to the reference (orremote) correlation analysis 62. In this case, all or most of thecorrelation analysis can be carried out by the reference correlationanalysis 62, on the basis of parameters abstracted from the batch 15 ofdata sets 1-11. The correlation optimization processing may of the datasets 1-11 may then be carried out by correlation optimization 23 or byreference correlation optimization 63.

As an illustration of how this method may be used in practice, we takean example of the Johnson family on holiday in Athens. Mrs. Johnsontakes many photographs of the children in front of the Parthenon, andwishes to upload the photographs to her account on a cloud-based imagemanagement server. Mrs. Johnson has a very good camera which takeshigh-quality photographs, each of which is stored as a very large file.However, internet access is slow and expensive, and the Johnsons wish toupload the image files from their camera to their server as quickly aspossible.

Luckily, many other families have previously taken photographs ofchildren in front of the Parthenon and uploaded them to the samecloud-based image management server. Many of the photographs are taggedwith the GPS location or with the tags “Parthenon” or “Acropolis”, orwith date, time or other information, and the server has performedextensive correlation analysis on the many photographs, and it hasgenerated a highly optimized set of correlation parameters, which itmakes available for encoding and uploading new images to the server.

The correlation optimization calculations can be performed by aprocessing unit local to Mrs. Johnson's camera (e.g. in the camera or onher laptop), or on the server, or on a different server, or on anycombination of such computing units.

When Mrs. Johnson's camera connects to the server, it may transfer abatch of images which have already been encoded in acorrelation-optimized fashion, with the correlation optimization basedupon the batch of images, or upon the batch of images and other imagesavailable locally. In an alternative mode, the camera may submit to theserver a set of parameters which can be used by the server to carry outcorrelation optimization calculations based on images stored on theserver, or available to the server. Such parameters may includeparameters extracted by analysis (e.g. by pattern recognition) from thephotographs. Such parameters also may include, for example, details ofthe camera settings or other parameters associated with each photographat the time the photograph was taken, or associated with the batchgenerally. Such parameters might include, for example, a GPS location ofthe camera, an angular/geographical orientation of the camera, a modeltype of the camera, a zoom setting of the camera lens, a time of day, anaccelerometer sensor measurement, etc. The server may then use some orall of this information, and possibly other information from othersources, such as prevailing weather conditions at the given time(s) andlocation(s), the angle(s) of the sun or moon at the time(s) andlocation(s), etc.), in order to select an image, or a set or images, ora pre-computed set of correlation optimization information, which canthen be used to calculate optimized correlation parameters forprocessing (uploading) the photographs.

The present disclosure may also be embedded in a computer programproduct, which comprises all the features enabling the implementation ofthe methods described herein, and which when loaded in a computer systemis able to carry out these methods. Computer program in the presentcontext means any expression, in any language, code or notation, of aset of instructions intended to cause a system having an informationprocessing capability to perform a particular function either directlyor after either or both of the following: a) conversion to anotherlanguage, code or notation; b) reproduction in a different materialform.

While the present disclosure has been described with reference tocertain embodiments, it will be understood by those skilled in the artthat various changes may be made and equivalents may be substitutedwithout departing from the scope of the present disclosure. In addition,many modifications may be made to adapt a particular situation ormaterial to the teachings of the present disclosure without departingfrom its scope. Therefore, it is intended that the present disclosurenot be limited to the particular embodiment disclosed, but that thepresent disclosure will include all embodiments falling within the scopeof the appended claims.

1-20. (canceled)
 21. A method for video processing, the methodcomprising: generating a plurality of correlation parameters, whereineach correlation parameter represents a degree of correlation betweentwo images of a plurality of images in a video sequence; ordering theplurality of images according to the plurality of correlationparameters; and delta encoding the plurality of correlation parametersaccording to the ordering.
 22. The method of claim 21, comprisingproducing the plurality of correlation parameters in accordance with oneor more correlation metrics associated with a delta encoding operation.23. The method of claim 21, comprising producing an optimizedsubdivision of the plurality of correlation parameters.
 24. The methodof claim 21, comprising producing two or more pluralities of correlationparameters, wherein each plurality of correlation parameters is producedin accordance with a different correlation metric.
 25. The method ofclaim 24, comprising producing an optimized correlation metric accordingto a comparison of the two or more pluralities of correlationparameters.
 26. The method of claim 21, wherein the ordering indicates amaximum of the plurality of correlation parameters.
 27. The method ofclaim 21, wherein the ordering comprises a first image followed by oneor more dependent images, wherein each of the one or more dependentimages are delta encoded according to the first image.
 28. A system forvideo processing, the system comprising: a processor that is operableto: produce a plurality of correlation parameters representing degreesof correlation between two images of a plurality of images, wherein theplurality of images are ordered independent of the plurality ofcorrelation parameters; reorder the plurality of images according to theplurality of correlation parameters; and delta encode the reorderedplurality of images.
 29. The system of claim 28, wherein the processoris operable to produce the plurality of correlation parameters inaccordance with one or more correlation metrics associated with a deltaencoding operation.
 30. The system of claim 28, wherein the processor isoperable to produce an optimized subdivision of the plurality of images.31. The system of claim 28, wherein the processor is operable to producetwo or more pluralities of correlation parameters, each plurality ofcorrelation parameters being produced in accordance with a differentcorrelation metric.
 32. The system of claim 31, wherein the processor isoperable to produce an optimized correlation metric according to acomparison of the two or more pluralities of correlation parameters. 33.The system of claim 28, wherein the reordering indicates a maximum ofthe plurality of correlation parameters.
 34. The system of claim 28,wherein the processor is operable to delta encode one or more dependentimages, wherein each of the one or more dependent images are deltaencoded according to a first image.
 35. A camera, the camera comprising:a memory for storing a plurality of images; and a processor that isoperable to: produce a plurality of correlation parameters representingdegrees of correlation between two images of a plurality of images;reorder the plurality of images according to the plurality ofcorrelation parameters; and delta encode the reordered plurality ofimages.
 36. The camera of claim 35, wherein the processor is operable toproduce the plurality of correlation parameters in accordance with oneor more correlation metrics associated with a delta encoding operation.37. The camera of claim 35, wherein the processor is operable to producean optimized subdivision of the plurality of images.
 38. The camera ofclaim 35, wherein the processor is operable to produce two or morepluralities of correlation parameters, wherein each plurality ofcorrelation parameters is produced in accordance with a differentcorrelation metric.
 39. The camera of claim 38, wherein the processor isoperable to produce an optimized correlation metric according to acomparison of the two or more pluralities of correlation parameters. 40.The camera of claim 35, wherein the reordering indicates a maximum ofthe plurality of correlation parameters, wherein one or more dependentimages are delta encoded according to a first image.